moneval_experts_2012_prioritizing_evaluation_questions

Edit 0 7 …

Africa RISING M&E Expert Meeting

5-7 September 2012
Large auditorium, ILRI Ethiopia, Addis Ababa
Back to the event agenda

Prioritizing evaluation questions


Comments:
  • Do we need to maintain the soil quality? We should improve it.
  • Some of these are development outcomes and others more about research outcomes - e.g. #1 will happen only later but #2 will happen earlier.
  • We also have to think about scale: dev impact are on a limited scale (e.g. partnerships etc.)...
  • Objective #1 is good but we're not likely to address this in the project - we should focus on monitoring outcomes that relate to our research.
  • These evaluation questions all hint at specific impact pathways. At the end of our program we end up with all issues.
  • This ranking doesn't mean that we are not going to address more than the top 3 issues.

1. Did Africa RISING Make a Contribution at the Goal/Highest Objective Level?
 8
(Ryan, Jerry, Tahirou, Joe, Jubal, Chris, Joseph, Naomi)
Did technologies or combinations of technologies have a positive impact on people’s lives, income, nutrition, livelihoods?
Did Africa RISING provide pathways out of poverty and/or malnutrition through agricultural SI? Which pathways did Africa RISING provide?
What is the contribution of SI to poverty alleviation, food security, nutrition, (livelihoods), NRM?
Did Africa RISING make a contribution in the context of multiple projects in the same geographic area?
2. Did Africa RISING increase agricultural productivity?
 9
(Irmgard, Sibiry, Joseph, Peter, Steve, Regis, Chris, Asamoah, Mateete)
Did Africa RISING increase whole-farm productivity? (labor, women’s labor, vs. single-crop productivity)
Did Africa RISING show how to best make use of available technologies?
Did Africa RISING interventions complement each other? (individual technologies vs. packages, synergies)
3. Are Africa RISING’s approaches replicable/scalable?
 10
(Sibiry, Naomi, Jerry, Jubal, Ryan, Joe, Pete, Tahirou, Joseph, Steve)
How scalable are the results of Africa RISING?
How big are the scalable impacts of Africa RISING?
What are the most successful elements of Africa RISING that should be scaled up/out?
Are the benefits of Africa RISING greater than the cost?
4. Did farmers adopt the new Africa RISING technology(s)?
 4
(Chris, Mateete, Peter, Regis)
Have Africa RISING technologies been taken up beyond the trial farmers?
Did Africa RISING learn why some farmers adopt and others do not?
5. What are the tradeoffs among different impacts?
 4
(Irmgard, Mateete, Steve, Ryan)
6. What is the variation in impacts across different domains, geographies, household types, & gender?
 5
(Joe, Irmgard, Tahirou, Naomi, Sibiry)
Which types of farmers did Africa RISING help? Are they the targeted ones?
7. Are the improvements made by Africa RISING sustainable?
 3
(Peter, Jerry, Asamoah)
Has water and/or soil quality been maintained?
8. Did Africa RISING demonstrate implementation methods that other projects can use?
 3
(Regis, Jubal, Asamoah)
Which processes/institutions were most helpful/useful in achieving improved outcomes? (Participatory vs. top-down)
How do we determine best-bet agricultural options for specific farmer types?
Did Africa RISING show how to use existing innovation platforms to scale out research efforts?
Did Africa RISING improve the efficiency of partnerships?
Are multiple stakeholder partnerships effective?
What is the optimal sequence of interventions?

Other questions:
  • Did AR collect data to complement existing data? --> Every project collects additional data.
  • Did AR bridge the gap among single-commodity projects? --> Not a major objective.
  • Which soils give the best response to mineral fertilizer? --> That's a research question.
  • Did we capture farmers' learning? --> that should be addressed as part of other evaluation questions.
  • How much did researchers learn? --> Custom indicator for researchers.
  • What did AR do that did not work? --> Fine, as long as you define how it didn't work (technically, institutionally).
  • Would AR technology disadvantage any segment of the community? --> This is part of the tradeoff package.
  • For scaling up, have we included a sufficient set of parameters for the projections to be credible? (different households will follow different pathways) --> That goes in the scaleability.

Other comments:
  • Is #7 sustainable? If we meet criteria by ?? the answer is yes, ditto with #2. We can more clearly rephrase this under sustainable intensification around a set of criteria and minimum SI levels. We could end up with one SI criterion lumping up a number of these criteria.
  • If we think about this being a research program, we need to see which objectives we can achieve for our program e.g. #2 and #7: that can be achieved in the project period. The list doesn't comply with some objectives of the whole program. Let's look at this first from a research point of view and then from a development point of view.
  • 3 & 4 seem related.
  • Adoption leads to productivity so we have to focus on productivity, not adoption. --> We need to focus on productivity under control. We need to show that our technology can lead to increased productivity first. We could also be testing technologies that elite farmers are practicing and asking questions related to not so privileged farmers. There are lots of options we could be testing. --> for some activities, you still have interactions with scientists so it's slightly controlled. We need to specify the range of productivity trials we can allow.